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Abstract. Visual communication, in the form of telephotography and 
television, for example, can be regarded as efficient only If the amount 
of information that it conveys about the scene to the observer ap- 
proaches the maximum possible and the associated cost approaches 
the minimum possible. Elsewhere we have addressed the problem of 
assessing the end-to-end performance of visual communication systems 
in terms of their efficiency in this sense by integrating the critical limiting 
factors that constrain image gathering into classical communication the- 
ory. We use this approach to assess the electro-optical design of image- 
gathering devices as a function of the f number and apodization of the 
objective lens and the aperture size and sampling geometry of the pho- 
todetection mechanism. Results show that an image-gathering device 
that is designed to optimize information capacity performs similarly to the 
human eye. For both, the performance approaches the maximum pos- 
sible, in terms of the efficiency with which the acquired information can 
be transmitted as decorrelated data, and the fidelity, sharpness, and 
clarity with which fine detail can be restored. 

Subject terms: electro-opticaf design: information; entropy: dynamic-range 
compression: image coding; image restoration. 

Optical Engineering 34(3), 795-813 (March 1995). 


1 introduction 

The problem of visual communication is that of producing 
an image that conveys information to the human observer at 
one point about a scene that is located at another point. Until 
recently, in telephotography and television, for example, the 
input terminal of the visual communication channel consisted 
solely of the image-gathering device that transforms the spa- 
tially varying radiance field reflected or emitted by the scene 
into the signal that is transmitted, and the output terminal 
consisted solely of the image display device that transforms 
the received signal into an image. However, advances in 
technology are leading to rapid growth in the capabilities of 
analog and digital VLSI processors, even as their cost, size, 
weight, and power consumption decrease. Consequently, vis- 
ual communication is now increasingly carried out by com- 
bining image gathering and display with digital image pro- 
cessing. Image gathering is combined with encoding to 
reduce data transmission, and image display is combined with 
restoration to enhance image quality. So far, however, the 
electro-optical design of image-gathering devices and the 
digital image processing for encoding and restoration have 
remained independent disciplines, following distinctly sep- 
arate traditions. 

The electro-optical design of image-gathering devices or- 
dinarily revolves around two interdependent trade-offs. One 
trade-off, in terms of geometrical optics, is widely under- 
stood. It deals with instantaneous lield of view (IFOV) versus 
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signal-to-noise ratio (SNR) as controlled largely by the ap- 
erture size and focal length of the objective lens together with 
the aperture size and responsiviiy of the photodetection mech- 
anism.'’^ Additional factors that may enter into this trade-off 
are field of view, depth of field, and exposure time.^ The 
IFOV and associated sampling lattice that evolve from this 
trade-off establish the angular resolution, or spatial scale, of 
the visual communication, ranging from telescopic to 
microscopic. 

The other trade-off, in terms of physical optics, w hich we 
address in this paper, is less widely understood. It deals with 
the relationship between the spatial -frequency response 
(SFR) of the optical (objective lens and photosensor) aper- 
tures and the sampling passband of the pholodetection mech- 
anism as a function of the SNR. The design that evolves from 
this trade-off establishes upper bounds on the information 
capacity of the visual communication channel at the spatial 
scale fixed by the sampling lattice. 

Traditionally, the relationship between the SFR and sam- 
pling passband of the image-gathering device has been op- 
timized to produce the best possible picture when the image 
display device reconstructs the received signal into an image 
without digital processing.^ For this reconstruction it is 
ordinarily preferred to let the SFR extend far beyond the 
sampling passband in order to avoid substantial blurring 
within the passband, at the cost of aliasing that the resultant 
insufficient sampling causes. However, the model of image 
gathering that has been used in the prevalent digital -image- 
processing literature^ takes account only of blurring and 
noise. The insufficient sampling has been ignored both in the 
assessment of digital image coding (e.g., by rate distortion 
theory) and in the formulation of digital image restoration 
algorithms (e.g., the Wiener filter). This failure to take full 
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account of the image-gathering process has seriously 
impeded the performance of visual communication systems 
in which image gathering and display are combined with 
digital image processing. 

To overcome this impediment. Pales and Huck have 
developed a mathematical foundation for assessing the end- 
to-end performance of visual communication systems by rig- 
orously integrating the critical limiting factors that constrain 
image gathering into the classical communication theories of 
Shannon'^ and Wiener.'^ These theories deal, respectively, 
wnth the efficient transmission of information through a noisy 
communication channel and the mininuim-mean-squared- 
error restoration of the input signal to the channel from the 
received signal. The inclusion of the image-gathering con- 
straints into these theories allows the designer to assess visual 
communication in terms of three intuitively attractive criteria: 
the information capacity of the image-gathering device, the 
theoretical minimum data rate al which the acquired infor- 
mation can be transmitted, and the maximum fidelity with 
which an image can be restored from the received data. It is 
an agreeable consequence of this approach*^' that the 
image-gathering device that is designed to maximize the in- 
formation capacity also tends to maximize the information 
efficiency with which decorrelated data can be transmitted 
and the sharpness and clarity as well as the fidelity w ith which 
images can be restored with the Wiener filter. 

In addition to the spatial properties of the radiance held, 
the designer must often deal with its wide dynamic range. 
Radiance fields in natural terrestrial environments exhibit 
dynamic variations up to 10" : 1, ranging from the darkest 
visible extended surface to the brightest patches that one 
commonly encounters.*"^ The dynamic range of photosensors 
is substantially lower. For photosensor arrays, such as charge- 
coupled devices (CCDs), the dynamic range is --10^ : 1, 
ranging from the noise level to the highest signal level. The 
preampliher that reads the signal out of the photosensor array 
to the analog-to-digital converter further limits the dynamic 
range to ~3 X 10^ : 1. The dynamic range of image display 
media is still lower. For film transparencies it may approach 
10\ 1, but for Him prints it is only - 10 : 1, ranging from a 
reflectance of ^0.8% to 80%. Within this narrow dynamic 
range it is possible to produce up to 64 perceptible gray levels, 
but one usually encounters less than 32 in practice. Hence, 
it is often desirable to compress the dynamic range of the 
acquired signal as early as possible in a way that best pre- 
serves the reflectance boundaries and topology of the scene 
while suppressing the temporal and spatial variations in the 
scene illumination. To this end, we combine image gathering 
with dynamic-range compression, similarly to the retinal pro- 
cessing in human vision. 

In Sec. 2, we present a mathematical model of the visual 
communication system together with the criteria that we use 
to assess its performance quantitatively. This assessment is 
mathematically limited to a linear (small signal) analysis; 
however, as we show later (in Sec. 4), the linear optimization 
has a significant effect on the performance when nonlinear 
transformation is added. Next, in Sec. 3, we characterize the 
electro-optical design of image-gathering devices. This in- 
cludes the human eye and recent developments of retina-like 
analog VLSI processors such as Mead's “silicon retina.’ 
The comparison of the informationally optimized electro- 
optical design with the early stages in human vision is ob- 


viously of interest because these stages are constrained by 
the same critical limiting factors as visual communication."* 
Finally, in Sec. 4, we evaluate the performance of the visual 
communication channel as a function ot the electro-optical 
design and the dynamic range compression, considering both 
image reconstruction and restoration. The difference between 
these two image representations is important: whereas re- 
construction is concerned w ith producing a continuous rep- 
resentation of the digital output of the image-gathering 
device, restoration is concerned with producing a represen- 
tation of the input to this device. 

2 Mathematical Model 

2.1 Visual Communication Channel 

Figure I depicts a model of the visual communication channel 
that combines image gathering and display, respectively, w ith 
digital encoding and restoration. Mathematical assessments 
of this model by communication theory are constrained by 
the assumptions that all processes are linear and isoplanalic, 
and that the radiance field and photosensor noise amplitudes 
are Gaussian, wide-sense stationary, and statistically 
independent. 

Image gathering transforms the continuous radiance Held 
L(.v.y) into the digital signal 

s{x,y) = I KLUy) t(.v,v) | ||l (.v,y ) T n^M^y) + /^/.v,y) . ( I a) 

where K is the steady-state gain of the linear radiance-to- 
signal conversion, T(.v,y) is the spatial response ot the image- 
gathering device, and /?^,(a,v) and n^^iwy) are the additive, 
discrete photosensor and quantization noise, respectively. 
The symbol -s* denotes convolution, and the function 

|||(,v,y) = XK X S 8(.v-X;)i.,v -)',() 

ff, = -V. fi — - 

denotes sampling in the ( v.y) rectangular coordinate system 
of the image-gathering process w ith sampling intervals (X K). 
This sampling lattice Hxes the spatial scale al which the com- 
munication channel operates. The treatment of quantization 
as an additive noise can be justified by Roberts' method for 
converting quantization noise to random noise of the same 
rms value.^' "* This conversion establishes a mathematical and 
perceptual equivalence between random and quantization 
noise. The Fourier transform of this process is 

.v(o,a)) = |XL(i),co)T{o,to)l * l||(o,a)) + n^,(o,o)) + /2 ^^(\),(d) , 

(lb) 

where />(\j,o)) is the continuous radiance-field transform, 
T(o,oj) is the SFR of the image-gathering device, //^,(o,cjd) and 
/?^^(o,o>) are the discrete noise transforms, and (o,a)) are the 
spatial frequencies with units of cycles per sample. The 
function 

||l(v.a))= 2 2 8|^v-^.a)-^j = 8(v,a)) + j|j,(uco) 

is the Fourier triinsform oFthe sampling latliee. where 8(v.co) 
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Image gathering Signal encoding 



Image restoration Signal decoding 

Fig. 1 Model of the visual communication channel. 


is the Dirac delta function and ||[ (u,(jj) represents the sam- 
pling sidebands. The associated sampling passband 




(U.O)), |u| < , \(X}\ < 


2X 


2Y 


has area |5| = I/XK 

The analog-to-digital conversion is done with k levels for 
T]-bit quantization, where t| = log 2 k. This conversion is sub- 
ject to the assumptions that the quantization is uniform, the 
error of any one sample is uncorrelated with that of any other 
sample, and the signal is equally likely to occur anywhere in 
the quantization interval —caj k to caj k. The two param- 
eters that define this interval are the variance o; of the ac- 
quired signal given by (disregarding the photosensor noise) 

= J j 4>^(ucjl))|t('u,(jo)P du do> 


where (o,o)) = < 7 / (u,o)) is the normalized PSD of the 

radiance field w ith variance <77 , Kdf kr^, is the rins signai-to- 
noise ratio (SNR) at the photosensor output, and (Kdj 
is the corresponding SNR for the quantization. As imple- 
mented here, this filter suppresses the blurring and raster 
effects of the image display process by interpolating between 
the acquired samples with a display lattice that is four times 
denser than the sampling lattice. This interpolation is depicted 

in Fig. I by the symbol III (xy). A denser image display lattice 
does not perceptibly improve the visual quality of the restored 
image, whereas a coarser lattice does reduce the visual qual- 
ity. Reference 22 presents a filter that minimizes these re- 
ductions in visual quality even if, in addition to a coarser 
lattice, the computational support is constrained to further 
reduce the complexity of processing. 

The PSD of the radiance field A(.v,y) within an 

isoplanatism patch A is assumed to be“^ 


and the constant c that adjusts the intervals of the quanti- 
zation. Consequently, the acquired signal is quantized over 
the dynamic range of to ro^, and signal values outsick 
this range are assigned to either 0 or k — 1 . We let v 3, 
for which the dynamic range encompasses 927< of the signal. 
This value of c relates information to data so that the theo- 
retically highest information efficiency (as given in Section 
2.2) is normalized to unity. Finally, it is convenient to 
assume that the photosensor noise has a white power spectral 
density (PSD) with variance <t‘. 

Restoration readjusts the magnitude of each sample of the 
digital signal .v(.v,y), interpolates between the samples, and 
transforms the digitally processed signal into the continuous 
image /?(xy), which has the Fourier transform 
/?(u,(d) = K~^ .v(D,to)'^(v,u>). The Wiener filter which 

minimizes the mean squared restoration error (MSRE) be- 
tween L(u,o>) and A(u,a>), can be expressed as'^' “ 

^(u,co) = 

(\j,cl))t*(v,<u) 

4);(u,CO)|t(U,CO)P *jj|(U,(0) + (Ar<7, /(7^,) - klA “K 

( 2 ) 


d>/(D,(0) 


277}JL“(77 

1 1 +(2tt|jlp)“| ^^“ 


(3) 


where p“=v“ + a>“ (Fig. 2). Figure 3(a) shows a target of 
random polygons that has this PSD. The target is generated 
by a Poisson process with mean distance pi (measured in 
sampling intervals) between radiance-field transitions and by 
an independent Gaussian intensity distribution with variance 
(T] at the transitions."^ The mean distance pi between the 
edges of the polygons is treated as the mean spatial detail of 
the radiance field. That is, when K= pi, the scale of the 
sampling lattice coincides with the mean spatial detail. Fig- 
ure 3(b) shows a target of resolution wedges that we include 
to facilitate assessments of the fidelity with which fine detail 
near the sampling lattice cun be restored. 


2.2 Performance Criteria 

Information theory treats the restored image R(x,y) as a re- 
ceived message that gives information about the radiance 
field L{x,y) and accounts for degradations as loss of infor- 
mation. In this sense, the information K can be expressed 
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Fig. 2 Normalized PSD of the random radiance field with 

mean spatial detail ^ relative to the sampling lattice. 







(o,a>)|T(o,co)|“ 

0;(o,w)|T(v,w)|“=f= ll|,(v,co) + (Afay - + {Kiif hr^)~-K ~~ _ 

Xci\J dco . (4) 

The information that the acquired signal contains, given 
here as the average per digital sample, is Shannon's rate of 
transmission of information, or information rate. 'M reaches 
its theoretical upper bound 


^ = 0.5 1.0 1.5 



(b) 


Fig. 3 Targets: (a) random polygons with mean distance |jl^ 3 per 
sample and (b) resolution wedges. 


|fi| 'og2l ■ + (A'(t, /<T,,)-| (5) 

when 


(0,(0) = 


t(u,(o) == 


{ 


0, elsewhere, 

1, (u,co)e B, 

0, elsewhere. 


and the quantization noise is negligible. This upper bound is 
Shannon's channel capacity, which is constrained only by 
the bandwidth B and SNR However, the image- 

gathering process inevitably bars from reaching W,,,, be- 
cause X is constrained also by the PSDs 4>/(o,(o) ot natural 
radiance fields and by the realizable SFRs t(u,(o) of optical 
apertures, both of which decrease smoothly with increasing 
spatial frequency. 

The information W for visual communicatmn reaches its 
maximum value when the sampling passband 5 best matches 
the radiance-field PSD d>, (o,a>). We designate this maximum 
value as the information capacity . Ordinarily, it occurs 
when the sampling interval is near the mean spatial detail p.. 
Comparisons of the information capacity with Shannon's 
channel capacity reveal that the image-gathering process 
constrains H\. to about half of 


The theoretical minimum data rate % associated with the 
information M is given by the mutual information between 
the acquired signal before quantization and its quantized rep- 
resentation.*^ It can be expressed as*^ 

< 4//. log J I 


<t>; (u,w)|T(v,to)|-*|l|(iJ,a)) + (A'cr, /a,,) - 
(KiTf /crj 


di) do> . 

(6) 


This expression for represents the entropy of completely 
decorrelated data. It sets the theoretical lower bound on the 
data rate that can be attained with lossless encoding. To dis- 
tinguish entropy from information, we measure entropy in 
binary units (bits), as is common for data, and information 
in binary information units (bifs). 

Note that the sampling sidebands (aliasing) and photo- 
sensor noise, which reduce X in Eq. (4), increase ^ in Eq. 
(6). Hence, approaches its lower limit X only when these 
distortions are masked by coarse quantization. About 80% 
to 90% decorrelation is obtained when either differential 
pulse code modulation (DPCM) or critically sampled miil- 
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liresolulion decomposition is combined with HnlTman en- 
coding.'^ The decomposition does not improve on the de- 
correlation produced by the computational I v much simpler 
DPCM. 

It is intuitively attractive to characterize the information 
efficiency of the image-gathering process by the ratio 
and to adjust this ratio so that its theoretical upper bound is 
unity. This adjustment occurs naturally for the conditions of 

the PSD 4Y(o,to) and the SFR t(o,o)) that lead from K given 
by Eq. (4) to its upper bound given by Eq. (3). These 
conditions are implictly included in Eq. (2) for 4^(u(o), Eq. 
(4) for li\ and Eq. (6) for i . An important result of earlier 
assessments,'^"'^ which we also show here, is that the electro- 
optical design that maximizes the information capacity 
for a given SNR also maximizes the information 

efficiency i 1. But there always remains a trade- olf, for 
each design, between )(^. and in the selection of the 

number of quantization levels. 

The fidelity F is a measure of the similarity between the 
radiance field L{.x,y) and the image F(.v.y) as delined by"^'^'"^ 

JJ,.' I ^(.Y. V) - R(.\.y) I - d-v dy 
//jL(,v,v)|- d.v dv 


The maximum value of F for the image R{x,y) restored by 
the Wiener filter is'^" 




’ll.. ,(u)7(o,a))4^(u,to) do, do> 

= jj 4>;(v,w)(l-2 dv> doj , 


( 8 ) 


where is the spectral information density given by the 

integrand of Eq. (4). Another important result of earlier as- 
sessments,'^"'^ which we also show' here, is suggested by the 
dependence of on 4f(o,(x)); namely, that the image-gath- 
ering device that is designed to maximize the information 
capacity Tf ordinarily can be relied on to maximize the fi- 
delity of the Wiener restoration. The apparent lack of reso- 
lution and sharpness of Wiener restorations is not, as was 
long believed, a shortcoming of the lidelity criterion.'^’ 
Instead, it is the failure of the traditional formulation of the 
Wiener filter, like that of other restoratit)n filters published 
in the prevalent digital image-processing literature.^ to 
allow fully for image gathering and display. When the Wiener 


filter fully includes these processes, then it produces images 
with improved resolution and sharpness as well as fidelity. 
Spatial details as small as the sampling interval are usually 
resolved. However, these images are also more sensitive to 
visual defects such as aliasing artifacts and photosensor noise. 
These defects diminish and the visual quality of Wiener res- 
torations improves in clarity with increasing information, 
even after the maximum fidelity has essentially been reached, 
until this improvement is gradually ended by the unavoidable 
compromise among resolution, sharpness, aliasing artifacts, 
and ringing. The visual quality may often be enhanced at a 
small loss in resolution and sharpness by combining the Wie- 
ner filter w ith a modest amount of smoothing. 


3 Electro-optical Design 

3.1 Design Parameters 

Image-gathering devices arc commonly specified by their 
IFOV and SNR. (To avoid possible confusion, it should be 
pointed out here that the SNR is given elsewhere more com- 
monly for the average rather than for the rms signal value.) 
These specifications can be injected into the image-gathering 
model given by Ex|s. (1) via the steady-state gain (in am- 
peres)'- 




0 - 


(X)r(\) d\= 


rrky~ 
^ 4F 


r f 

J o 


/,(X)/ (\) d\ , (9) 


where Fi ^irD^/4 is the area of the objective lens aperture, 
= steradians (sr) is the solid angle of the 

IFOV, and F-- fll) is the lens / number. Other design pa- 
rameters are identified in Fig. 4 and Table I . The SFR t(u,o)) 
of the image-gathering device is the product of the SFR 
T^iuoi) and t^,(d,o>) of the objective lens and phenosensor 
a pe rt u re , re sped i v e 1 y ; i .e . , 


t(uoj) = T,(U.OJ)T^,(n.O>) 


( 10 ) 


The objective lens is characterized in Sec. 3.2 by its dif- 
fraction-limited performance. However, few real lenses 
achieve this performance over their w hole held.“‘'*'^" The pho- 
tosensor array is chai acterized in Sec. 3.3 for the square and 
hexagonal sampling lattices, and lateral inhibition wnth these 
tw'o arrays is characterized in ,Sec. 3.4. It is common, as we 
do here, to characterize the objective lens by its coherent 
cutoff frequency 1/2XF and the optical geometry that the 


Picture 

element 


Objective 

lens 


Photosensor 

array 


Y 



Fig. 4 Optical configuration of the image-gathering device with photosensor array. 
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Table 1 Design parameters. 


l 'ammeter 

Symbol 

Unit 

Limis Hporturo diameler 

D 

m 

I, CHS focal length 

f 

m 

l^hotosensur aperture spacing 

X 

m 

Photowuisor aperture width 

1 

rn 

Photosensor distance from lens 

fj, 

rn 

Wavelength 

i 

fim 

Pilot osens( >r r es p< iiisi v i t y 

r(A) 


Spectral radiance 

L{X) 

Wrn ^ sr * /im ^ 


objective lens forms with the phoiosenscM' aperture by the 
optical design index y/2XF. 

3.2 Objective Lens 

The SFR, or normalized optical transfer function (OTF), 
T^(uoj) of a diffraction-limited lens with a circular aperture 


T^(uoy)^ 


J/" , F/iv ^ )Fi{v - pflM)' ) cxpiiupv) di)', dto' 




Fjiv\i)}' )Fi(v\ii)' ) dv dw' 


( 11 ) 


where 




/(uoj), 

0 


v^w-^1 , 
elsewhere. 


p = iv~ + ii}-) "2KF , 

and is the transmittance. The dimensionless spatial- 

frequency variables F, to, and p are normalized to the coherent 
cutoff frequency 1 /2XF. The defocus parameter u is 


77 

M 

2\ 



77 A f 

2\F~ 


where A f - | ( , — \ , and i) is the image-plane distance from 

the lens (Fig. 4). 

Hopkins^' has formulated the SFR of a defocused dif- 
frac t i o n - 1 i m i ted 1 e n s w i t h a c I ea r a pe rt u re , i , e . , / (i), to ) = 1 , an d 
Mino and Okano^^ have extended this formulation to include 


two circularly symmetric variable lens transmittance shad- 
ings that reduce defocus blur. OTFs for different shadings 
also can be obtained directly by numerical integration t)f Eq. 

( 1 1 Figure 5 shows the SFR T<(tuo) for a clear and two 
shaded apertures. The shadings are given by 

Mti.w)=l-p^^ (12) 

with p = 1 and 2. The ratio k of light transmitted through the 
shaded aperture to a clear aperture, i.e., the effective trans- 
mittance in Eq. (9), is 



As can be seen, the shading reduces the effect of defocus on 
the SFR of the objective lens. However, this improvement 
can be gained only at a loss of transmittance. 

The angular sensitivity of the eye's photoreceptors (i.e., 
the Stiles-Crawford effect ^^9 produces a similar result to the 
variable lens transmittance shading. Metcalf^^ has shown that 
the angular sensitivity of the photoreceptors can be consid- 
ered as equivalent to a variable pupil aperture transmittance, 
and Carroll^^ has shown that this variable transmittance pro- 
duces a SFR that is similar to the one given by Fig. 5(b) for 
P = 2. However, the advantages of this angular sensitivity, 
or its equivalent aperture shading, have apparently not been 
widely recognized; namely, that it enhances the SFR within 
the photoreceptor sampling passband and renders the SFR 
less sensitive to defocus. 

3.3 Photosensor Arrays 

Figure 6 depicts two photosensor arrays characterized by 
T^4.v,y)|||(.v.y) and T^,(uco)'.}:|||(v.a}). For the conventional 
square array^"* 


, l-'i<-y/2, |.v1<7/2 . 

|() elsewhere. 

(14a) 

T^yu.to) — sincyi) sincyco. 

(14b) 

1 

1 



u 



Fig. 5 SFRs of diffraction-limited lens with clear and shaded apertures for a coherent cutoff frequency 
1/2XF- 1 and several values of the defocus u: (a) clear aperture, k- 1, (b) shaded aperture, p - 2 
and k^O.33, and (c) shaded aperture, 1 and k 0.17. 
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(15b) 


and for the regular hexagonal array 
1 


0 elsewhere. 


— , V <— , Lv +- V <- 


(16a) 


1 . 


ir J V 

cos— 7 — :r — (U 

2^ VV3 


• ,( ^ 

X sinc-y 1 — to 
2 V V 3 


+ COS— y — ^ + (0 I sinc-y 

2 Vv3 / 2 


I , / x> 


V3 


+ COS 


TT7 V 

V'3 


Xsinc — y I — ^ — u) I sinc -7 — ^ + (ji) 

2'Vv3 / 2'\v3 


(16b) 


■(.,-.y)=^r= i i 


in ~ — y- n = 


, V -2 , 1 , 

X 8 (.v —X (m + n).y — -X {ni — n) 


(17a) 


in -- y u - 


||'(v,(o)== 2 ! S ^ 777 '^ 77; — I ■ (12b) 


V3a:' 




of the square lattice has area 1 ^| = 1 /X^, and the sampling 
passband 



(i^,co); |u| 


1 H ^ 1 1 

\ 3X'’ 2 2 V3X'J 


of the hexagonal lattice has area \ B'\ = 2/ V3X'^. The areas 
1^1 and 1 ^' |, and hence the sampling densities of the square 
and hexagonal lattices, are equal to each other when the 
dimensions X and X' are such that X = (\/3/2)^^ 
X'=0.93X'. 

Past comparisons of the square and hexagonal lattice have 
been based on the premise that the signal spectrum is cir- 
cularly symmetric and bandlimited.^'^‘^‘^ Given this premise, 
it has been shown that the hexagonal lattice optimizes the 
density wdth which the circularly symmetric sidebands can 
be arranged in the spatial -frequency domain without overlap, 
permitting exact reconstruction of the signal with a minimum 
number of samples. In particular, Peterson and Middleton 
have shown that the contiguous sampling sidebands cover 
90.8% of the spatial-frequency domain for the hexagonal 
lattice, compared w ith only 78.5% for the square lattice. Con- 
sequently, as Mersereau^^^^ has shown, the hexagonal lattice 
can have 13% fewer photosensors (and correspondingly 
larger photosensor areas) than the square lattice for the exact 
reconstruction of circularly symmetric, sufficiently sampled 
signals. 

3.4 Lateral Inhibition 

Figure 7 depicts a hypothetical mechanism that adds lateral 
inhibition to the two photosensor arrays. The diagram shows 
only those interconnections that form a single neighborhood 
subtraction. The subtraction is performed with either eight 
neighboring phoU^sensors for the square array or six for the 
hexagonal array. For digital VLSI implementation, many 
more interconnections are required so that each photosensor, 
in effect, serves once in the central position and either eight 
or six times in the neighboring positions. However, for analog 
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Fig. 6 Photosensor arrays: (a) square and (b) hexagonal. 
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Fig. 7 Photosensor arrays with lateral inhibition: (a) square and (b) 
hexagonal. 
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VI. SI implcmciUalions, this circuit can be implemented with 
a relatively simple resistive network. 

For the square array, the spatial response becomes"^' 

[ where is given by Eq. (18a). The noise is now 

y) replaced by 


V,V,V)=/\(0-i T„( 


+ 2 cosTT X ' ( V + (0 ) + 2 COSTT.V ' ( V 3\) — to ) ] j , 

(19b) 



where ^ (O^^^l ) is the lateral inhibition index and 
is given by Eq. ( 14a). The coiTcsponding SFR is 

T^,,(uco) =A(^)t^,(u( 0 )| I -4a^(l -cos“7TAre-cos“TTXto) 

+ 4/;^( 1 —2 cos"7tXu)( 1 - 2 cos“7tA'o))| , 

(18b) 

where T^,{u,to) is given by Eq. (14b). Only the form of 
T^,,(o,to) should be used to characterize the SFR of the pho- 
tosensor array with lateral inhibition. The factor /l(^) must 
be associated w ith the photosensor and aliased noise because^^’ 
the normalization of T^,,(u,to) is intimately tied to the gain 
constant K. The SFR t^,,(u.(o) most closely approaches cir- 
cular symmetry when a = 0. 1 8 and b = 0.07. 

If electronic noise sources other than the photosensor are 
disregarded, then the output noise level is 




' + U+^ 


Both the signal power and the aliased noise power are pro- 
portional to |/t(^)|“. Therefore, the gain ^he SNR 

expression KiTflUf, cancels, and one may choose any nor- 
malization factoiM(0 ns long as the noise is replaced by 
the Here we let 1. Hence, the increase in noise 

that lateral inhibition causes for this hypotethical circuit is 
at most a factor of 1.13 when ^ = 1 . 

Similarly, for the hexagonal array, the spatial response 
becomes'^* 


T.,,(.v,y) = /\(0 


T;,(-v.y)-- 2 


ni.n i 
{nt it N) 


■ , V3 , X' 

T,,|.V ^X (w + /i), ,\ —{m-n) 


, y^X. 

(19a) 


where T,',(.v.y) is given by Eq. { 16u). The corresponding SFR 
is 


7 , ( \J,CO ) = A ( ^ ) T ^',( U,tO ) 



cos(2TrXTo) 


<r,„ = 0(Ol(l -(T,, , 

w'hich for /\(5) = 1 is at most a factor of 1.17 when ? - 1 . 

3.5 Information Capacity 

Figures 8 and 9 characterize the information capacity as 
a function of the optical design index y/2\F for several 
SNRs A'cr/Zoy. The noise variance (t^, includes both pho- 
tosensor and quantization noise. The objective lens is clear 
in Fig. 8 and shaded in Fig. 9. Both tigures provide for the 
square and hexagonal photosensor array lattices, either w ith- 
out lateral inhibition (c=^0) or with maximum lateral inhi- 
bition (^ = 1 ). 

The curves show that }(\ depends critically on the optical 
design index 'y/2\F. The information capacity is irrevocably 
constrained to be bifs, regardless of the SNR, however 

high it may be, if the optical design index for a diffraction- 
limited objective lens falls outside the range ().3<7/2AF 
<0.6. Within this range, the information capacity reaches its 
maximum value lor the SNR K<t j / a s, = 256 when y/2kF 
=^0.3.^ for the clear lens and when 7/2XF~0.4 for the shaded 
lens. Then, bifs w ithout lateral inhibition and lt^ ^4.5 

bifs with maximum lateral inhibition. Ji\. is a little lower than 
these values for the clear aperture and square sampling lattice, 
and it is a little higher for the shaded aperture and hexagonal 
sampling lattice. However, .H'\. depends tar more on the op- 
tical-design index and the SNR than on the lens shading, 
sampling geometry, or lateral inhibition. 

To characterize the dependence of on the electro-op- 
tical design in more detail. Figs. 10 and 1 1 illustrate the 
responses of the image-gathering device for the square and 
hexagonal arrays, respectively. As can be seen, the SFRs of 
the photodetection mechanism extend far beyond the sam- 
pling passband h even for contiguous apertures. Therefore, 
to optimize the information capacity , the objective lens 
must suppress the out-of-band portion of the SFR. The exact 
amount of suppression becomes increasingly critical as one 
tries to improve , by increasing 

Figure 12 shows that Gaussian responses closely approx- 
imate the SFRs T(n,co) of the image-gathering device. This 
allows us to model the SFR of image gathering with lateral 
inhibition by the difference-of-Gaiissians (DOG) function 






- ; exp 


-d.6p/e)' 


( 20 ) 


where, as Fig. 1 3 depicts, ^ is the optical response index that 
controls the relationship between t(\),co; 6.0 F(i.e., the 

trade-off between aliasing and blurring), and ^ is the lateral- 
inhibition index that controls the dynamic-range compres- 
sion. The DOG response for I = 1 closely approximates the 
Laplacian-of-Gaussian (V“G) response, where = 

+ d“/flv^ is the Laplacian operator. This operator, in effect. 
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Fig. 8 Information capacity a function of the optical-design 

index 7 / 2 for several SNRs The results are given for a 

clear objective lens and the (a) square and (b) hexagonal photosen- 
sor arrays with contiguous apertures. The lateral inhibition is either 
C-O or 1 . 


enhances the radiance -fie Id transitions after those transitions 
have been smoothed by the (approximately) Gaussian re- 
sponse of the image-gathering device. 

3.6 Human Vision 

Early vision is constrained by the same critical limiting fac- 
tors as visual communication. The resolving power (or sam- 
pling passband) of all eyes, in invertebrates as well as in 
vertebrates, appears to be limited by the difficulty of confining 
light within the outer segment of a photoreceptor.*'^ Center- 
to-center spacings of foveal cones less than 2 jxm have never 
been found, regardless of the size of the eye.^“ Moreover, 
the pupil / number remains remarkably constant in a variety 
of species, ranging from humans to birds of prey of widely 
different sizes. The sensitivity of the eye appears to be 
limited by the thermal stability of the photosensitive pigments 
of the photoreceptors. Because '‘dark’' noise is extremely 
low, equivalent to only a few quanta of light, the sensitivity 
essentially is limited by photon noise rather than the statis- 
tically independent noise assumed in this assessment. 

The human eye as a whole can encompass the wide dy- 
namic range of the natural radiance fields that one commonly 
encounters. If one allows for changes in pupil (or lens) di- 
ameter and for optical losses, then the dynamic range of the 
radiance incident on the retina reduces to ^ 10^ : 1 . Barlow''^ 
estimates that the number of distinguishable intensity levels 
in this dynamic range is ^200. Retinal processing seems to 
reduce this number of levels by a factor of to the upper 
limit of —40 levels that each nerve fiber can transmit from 
the retina to the visual cortex within —1/20 s to avoid pro- 
longing the reaction time. 

The eye’s pupil diameter is l)-2.5 mm in bright light, 
and the effective focal length is/= 17 mm (i.e., T = 6.8). The 


Kajo^ 


256 





Y/2XF Y/2XF 

(a) (b) 


Fig. 9 Same as Fig. 8 , but for a shaded objective lens with 
The symbol • marks the performance of the human eye. 


photoreceptor spectral responsivity is centered around 
\ = 0.56 fxm, and the width of the contiguous foveal cones 
arranged in a hexagonal lattice is y ' = 3 fxm. Hence, the IFOV 
77 /=^ 0.18 mrad =0.01 deg, and the optical design index 
772\f' = 0.4. Finally, the angular sensitivity of the eye’s 
photoreceptors produces a similar effect to the lens ap- 
erture shading for (3 ~ 2. 

The number of distinguishable levels, l, can be related to 
the SNR /C(r/ /(T,v by '^ 

i = |l -F(/L.r,/<r,,r|' ' . (21) 


According to this relationship, the l = 200 distinguishable 
levels are equivalent to a SNR of ^o-, /<t^= 256 (for |ul = I 
and ^ = 0.3).'^' Hence, there exists an intuitively appealing 
correspondence between the design of the human eye and 
that of an image-gathering device that is informationally op- 
timized, as characterized in Fig. 8(b) for 772Xf’ = 0.4, 
K<Ji 256. It follow s that the eye's information capacity 
is — 4.5 bifs (or 23 information levels). This performance 
is robust to changes in pupil diameter. As diffraction blur 
decreases with increasing diameter, aberrations increase. 
This compensation has not, to our knowledge, been exploited 
for digital cameras, even llK)ugh it could improve their per- 
formance and occasionally also reduce the cost of their optics 
simply by relaxing constraints on aberrations. 

If the DOG function given by Eq. (20) is used as a basis 
for modeling the angular response of the human eye, then its 
response for the optical design index 772\F= 0.4 is the one 
shown in Fig. 14. This DOG function is the best-known model 
of receptive Melds and has been used successfully to model 
the spatial and spatial -frequency responses for individual 
neurons"*''-^^’ in both the retina and the lateral geniculate nu- 
cleus (LGN). Measurements ol' the highest-resolution pro- 
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Fig. 10 Spatial responses and SFRs of the image-gathering device with the square photosensor array 
and the clear objective lens. The optical design index 7 / 2 0.42. 


cessing in human vision have been limited to anatomical and 
perceptual data, neither of which can provide direct infor- 
mation about the angular response. The gap in physiological 
data is of necessity tilled by other primate studies, especially 
of the macaque monkey, w^hose vision is considered to be 
similar to human vision. Moreover, these studies of neuron 
responses have emphasized the LGN, i.e„ the target of the 
optic nerve tract from the retina rather than the retina itself. 
However, it seems reasonable to assume that the highest- 
resolution neurons in the retina, the midget ganglion cells, 
correspond directly to their counterparts, the parvocellular 
layers of the LGN, and that both of these are the neural 
structures responsible for the highest-spatial -resolution pro- 
cessing in hu/nan v/.sion. 

Anatomical measurements of neural structures in the retina 
indicate that the IFOV per cell is 0.023 deg,"^^ which cor- 
responds closely to the center diameter of the DOG model 
in Fig. 14(a). Physiological measurements of primate retinal 
responses are consistent w ith this center dimension, but differ 


about the shape and extent of the surround response. One 
type of surround resembles the DOG response based on near- 
est neighbors but has an additional positive annular response. 
The other type has a very w ide weak surnuind about 30 times 
larger than the center. Of course, the surround response does 
not affect the finest detail that can be resolved, but rather 
deals with the lower spatial frequencies. It is, therefore, 
noteworthy that the second response, with the wide but weak 
surround, when coupled with a log intensity response, be- 
comes a central element in several lightness theories.^'^ 
Physiological measurements of contrast sensitivity and 
dynamic range at the highest resolution of processing have 
been largely confined to the LGN neurons. Most studies are 
consistent in finding that the contrast sensitivity for the parvo 
cells is only about 10:1 but covers a relatively large dynamic 
range. Perceptual measurements indicate a contrast sensi- 
tivity that is even coarser at the highest spatial frequency, 
only about two shades of gray,’'' but at lower spatial fre- 
quencies approaches 200 ; 1 . Recent measurements also sug- 
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Fig. 11 Same as Fig. 10, but for the hexagonal photosensor array and the shaded objective lens with 
(3-2. The optical design index 7' /2 \ F- 0.34. 
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Fig. 12 Informationally optimized SFRs for three SNRs: (a) (b) 64, (c) KtrJ 

«r/^ = 256. The SFRs are given for the clear and shaded 0 = 2) objective lens and the square photo- 
sensor array with contiguous apertures (7 - X= 1). Also shown are Gaussian approximations of these 
SFRs. 
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Fig. 13 Difference-of-Gaussians (DOG) SFRs for (a) ^^0, (b) ^ = 0.3, (c) ^ = 0.4, and (d) ^ = 0.8. 



x,y (degree) 

(a) 
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Fig. 14 DOG model of human visual response: (a) angular response 
in degrees, and (b) angular frequency response in cycles per de- 
gree. 


gest a variety of nonlinear interactions in visual processing 
in addition to the log intensity response, which provides dy- 
namic-range compression, adaptive contrast, and contextual 
sensitivity.*'" 

3.7 Analog and Digital VLSI 

Foca}-phine processing may be implemented in a number of 
different ways, if we take the term to mean processing within 
a camera but not necessarily within the image plane proper. 
Pure image-plane processing is realized in Mead's “silicon 
retina," where photosensor territory is sacrificed for pro- 
cessing territory with an analog VLSI resistive network. 


This disadvantage can be overcome by moving the analog 
processing to a separate chip.'’^ This move allows the pho- 
tosensor apertures to remain contiguous, but it reduces the 
effective dynamic range on account of the amplifier that is 
required to read the acquired signal off the photosensor array 
chip. Very recently, analog VLSI capacitor networks have 
been exploited^^ in connection with IR imaging systems. 
Significant advances have been made in the rapidly devel- 
oping technology of analog VLSI image processors since the 
last review^^^ of its history. Digital VLSI processors, by com- 
parison, sacrifice the power and wiring economies inherent 
in analog VLSI by orders of magnitude,''^ but gain in flex- 
ibility, stability, and accuracy. 

Thus far, analog VLSI implementations have considered 
only spatial summations and differences, and a limited set of 
nonlinear intensity transformations (mainly logarithmic). 
Whether this technology will prove to be flexible enough to 
encompass other vision computations is a key question for 
future research. Perhaps the future of analog VLSI will be 
best guaranteed if it proves to be capable of the computations 
that provide a sort of universal front-end processing that sup- 
ports a wide array of subsequent more specialized visual 
computations. To the extent that analog VLSI can mimic 
early natural vision, some confidence is justified that it is 
providing a generic foundation upon which a variety of di- 
verse vision systems can be based. Even if analog VLSI is 
successful for universal vision-primitive computation, digital 
processing will almost certainly enter the system design for 
subsequent processing. 

4 Performance 

4.1 Small-Signal Linear Analysis 

Figure \5 characterizes the information capacity and the 
maximum realizable fidelity as functions of the SFR 
^nd the SNR KiJiJijjy at the photosensor output 
(i.e., before quantization). The SFR, in turn, is controlled by 
the optical-response index ^ and the lateral-inhibition index 
^ (Fig. 1 3). As the curves for reveal, the information 
capacity depends critically on ^ (i.e., on the trade-off between 
aliasing and blurring) as a function of the SNR, consistent 
with its dependence on the optical-design index yllKF as 
shown in Figs, 8 and 9. This dependence becomes now in- 
tuitively appealing in terms of the restoration of fine detail 
near the sampling lattice. At one extreme, when the SNR is 
low, one would prefer to avoid substantial blurring because 
the noise constrains the restoration. At the other extreme. 
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Fig. 15 Information capacity rf and maximum realizable fidelity J as functions of the optical-response 
index i; for several SNRs The lateral inhibition is (a) ^-0, (b) C -0.6. (c) ^==0,8, and (d) 


when the SNR is high, one would prefer to avoid substantial 
aliasing because the noise is no longer a constraint. 

The curves for also reveal that the loss of information 
w ith increasing lateral inhibition is small for image-gathering 
devices with SFRs (e.g., ^=^0.3) and SNRs ( /(F|, = 256) 
that provide a high information capacity, but that it is large 
for devices with the conventional SFR (i.e., ^ — 0.8) regard- 
less of the SNR. Finally, the curves for both Tf and ) reveal 
that the optical design that maximi/cs }(\ also maximizes the 
fidelity 1 . Again, the loss of fidelity, like that of information, 
is small for a device w ith a high H\ and comparatively large 
for a device w ith a low i/f . Therefore, it is for the conventional 
design that one can expect lateral inhibition to impair the 
restoration of fine detail most severely. 

Figure 1 6 characterizes the information capacity and 
information efficiency )i^ f I as functions of the number of 
cjLianti/ati(^)n levels. Lateral inhibition is not included. The 
selection of quantization levels given in Table 2 for the two 
informationally optimized designs favors information capac- 
ity. A smaller number of quantization levels for these two 
designs would increase the information efficiency, but only 
at the cost of information and fidelity. However, for com- 
mercial digital imaging systems, as characterized by design 
I , the 8- bit quantization could be reduced to 5 -bit quantization 
to increase the information efficiency without loss in the 
information capacity. In practice, the preferred r|-bit quan- 
tization can be matched to the standard 8-bit (t)r 1 2-bit) format 
of commercial codec (encoder/decoder) methods, such as 
JPEG, simply by assigning the coarse quantizations fonq = 7, 
6, and 5 to every second, fourth, or eighth level, respectively, 
of the 8-bit standard. 

Figure 1 7 characterizes the dynamic range compression 
the information capacity and the information 


efficiency #, // as functions of the lateral-inhibition index 
^ for the three designs specified in Table 2. The dynamic 
range compression is given by the ratio 

<’'< H' X T/(v.n))|'r(ua);^.C)r di) do) ^ 

<r.|, y |t(d.<i);^,())|' di) dto 

By suppressing the low' SFR, lateral inhibition decreases the 
within-passband signal components relative to the aliased 
signal components (see Fig. 1 3). Consequently, the interfer- 
ence from aliasing increases strongly as the optical response 
index ^ increases beyond 0.4 w hile the lateral-inhibition index 
^ approaches I. It is therefore not surprising that, as lateral 
inhibition increases, ^ and )i ^ f I decrease significantly for 
design I, modestly for design 2, and negligibly for design 3. 
For design 2, ^ == 0.7 provides a dynamic range compression 
~4 and reduces the information capacity to 7(^ —3 bifs; and 
for design 3. the lateral inhibition with ^ = 0.8 provides a 
dynamic range compression ^5 and reduces the information 
capacity to 7f —4 bifs. Both designs 2 and 3 represent rea- 
sonable alternatives; the choice between them depends 
mostly on the SNR that can be attained. 

Figure 1 8 presents Wiener restorations for the three de- 
signs specified in Table 2. The lateral-inhibition index is 
either ^ = 0 or 0.8. Consistent with the curves for )(, and 7^ 
in Fig. 1 5, design 2 significantly improves on the resolution 
and clarity attained with the conventional design I . The im- 
provement in visual quality for design 3 over that for design 
2 is relatively small. This improvement requires significantly 
higher SNR but not a higher data rate. Note, in particular, 
that the lateral inhibition does not perceptibly impair the 
visual quality for the two optimized designs, w hereas it im- 
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Fig. 16 Information capacity ft ^ and information efficiency HJf as 
functions of the number of quantization levels q. 


Table 2 Characteristics of conventional and informationally opti- 
mized image gathering for |x = 1 . 


Design 


K ffjotp 

rj, bits 

Tic* bife 

£, bits 

Hcie 

1 Conventional 

0,8 

16 

8(5)* 

1.8 

7.5(4,6) 

0.24(0.39) 

2 Optimized 

0.4 

64 

7 

3.3 

5.6 

0.59 

3 Optimized 

0.3 

256 

8 

4.2 

5.6 

0.75 


• 5-bit quantization may often be sufficient for the low information capacity He of this 


design. 


pairs the already poorer visual tjualily tor the conventional 
design even further. Moreover, the theoretical mininuini data 
rate for the two optimized designs is a factor of 1.3 lower 
than for the conventional design, Thus, the informationally 
optimized design offers a lower data rate in addition to better 
image quality and more robust dynamic-range compression. 

4.2 Large-Signal Nonlinear Analysis 

Image gathering with nonlinear conversion can he included 
in Eqs. ( 1 ) by replacing the linear gain K with the nonlinear 
gain K{(x). In practice, the nonlinearity is commonly intro- 
duced in the analog-to-digital converter, i.e., after sampling 
rather than before. However, the order of nonlinear conver- 
sion and sampling can be interchanged mathematically. 
Moreover, the nonlinear conversion affects mostly the w ide 
dynamic range of the low-spatial-trequency components ot 
the radiance held without significantly distorting the high- 
spatial-frequency detail that one wishes to enhance. There- 
fore, in practice one can often interchange the order ot non- 
linear conversion and spatial convolution without introducing 
serious errors.'*'^ 

A common nonlinear conversion is given by the power 
relationship^'^ 

, (23) 

which is illustrated in Fig. 19. This nonlinearity increases the 
quantization density in the dark portions of the radiance held 
at the expense of reducing it in the bright portions. For a 2. 
this relationship becomes the “square -root ' coding often 
found in image encoding and models of nonlinear responses 
in human vision. 

Figure 20 depicts three irradiance profiles that span the 
width of the random target. The variation in the irradiance 
is such that the average radiance held, as it would he measured 
by a light meter, remains constant. The irradiance of the 
shadowed region is a factor of either 5, 25, or 1 25 lower than 
that of the bright region. These ratios encompass the varia- 
tions in the depth of terrestrial shadows that one commonly 
encounters. 

Figure 21 characterizes the radiance held that is the prod- 
uct of the reflectances of the random target and the three 
irradiances. It also characterizes the corresponding histo- 
grams of the radiance held and acquired signal, and the im- 
ages restored from this signal. Results are given for design 
2 as specihed in Table 2. The restorations include histogram 




; c c 

(a) (b) (c) 

Fig. 1 7 Characteristics of image gathering with lateral inhibition for the three designs specified in Table 
1; (a) dynamic range compression, (b) information capacity, and (c) information efficiency. 
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Fig. 18 Wiener restorations for (a) design 1 , (b) design 2, and (c) design 3 characterized in Table 1. 
The amount of lateral inhibition is t = 0 for the upper two rows and £ = 0.8 for the lower two rows. 


equalization and edge enhancement. The latter is obtained j^e deep 125 : I shadow is compressed into the narrow 
by letting the SFR t(v,w; 4.C) in the Wiener filter given by |0 ; I dynamic range of the image display. 

Eq. (2) be = 0) regardless of the value of ^ that is It may be noted that the SNR for the average value of the 

actually used. In general, it is desirable to increase both non- spatially varying irradiance was kept the same as for the 

linearity and lateral inhibition as the shadow deepens. It is uniform irradiance, i.e., /Ta/ /tr^, - 64. Consequently, the ac- 
then possible to resolve much of the structure of the scene tual SNR in the shadowed region is a factor of 3 lower than 

in the displayed image, even when the radiance tield with this value for the 5 ; I shadow, 13 for the 25 : 1 shadow, 
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Fig. 19 Nonlinear radiance*to-signal conversion for k = 256. 


and 63 for the 125 : I shadow. For the deepest shadow, 
therefore, the SNR is only about unity. Yet the visual quality 
of the restored images is not perceptibly impaired by noise. 
This apparent robustness to noise results from the dynamic- 
range compression of the noise as well as of the signal var- 
iations by the nonlinearity of the film, which compresses the 
contrast of dark and bright levels near the limits of its dynamic 
range. However, if the spatial details in the shadowed region 
are displayed with improved contrast (at the cost of over- 
exposing the bright region), then the need for a higher SNR 
becomes immediately apparent. 

5 Concluding Remarks 

The electro-optical design of the image-gathering device sets 
an upper bound on both the angular resolution and the in- 
formation capacity of visual communication systems. The 
information capacity, in turn, sets upper bounds on the ef- 
ficiency with which information can be transmitted and the 
visual quality with which images can be restored. 

The electro-optical design trade-offs involve a large num- 
ber of design variables and hence a wide range of design 
options. Nevertheless, if these trade-offs are optimized along 
guidelines developed from communication theory, then they 
lead inexorably to a design that is very similar to that of the 
human eye. The performance with this design approaches 
the maximum possible in terms of (I) the information ca- 
pacity ^5 bit's), (2) the information efficiency (Xit. =0.8 
bit's /bit) of decorrelated data, and (3) the fidelity, sharpness, 
and clarity of the restored image. If lateral inhibition, akin 
to the retinal processing in the human eye, is included to 
reduce the wide dynamic range of natural radiance fields, 
then the information capacity for this design diminishes only 
slightly to =4.5 bits. 

It may often be well worth while to use some of the clever 
mechanisms that the human eye employs to adapt to the wide 
depth of held and dynamic range that it commonly encoun- 
ters: 

I. As the pupil diameter increases to compensate for a 
decrease in scene illumination, aberrations increase to 
counter the associated decrease in diffraction blur. Our 



Fig. 20 Irradiance profiles across the random target. 

results show that this increase in aberrations helps to 
maintain the informationally optimized relationship 
between the SFR of the optical (pupil and photorecep- 
tor) apertures and the sampling passband of the pho- 
toreceptor lattice. 

2. The angular sensitivity of the photoreceptors enhances 
the SFR and its robustness to defocus. The same en- 
hancement can be produced by shading the objective 
lens with a variable transmittance. 

3. The lateral inhibition in the retina acts like a second- 
order differentiator to enhance the radiance-field tran- 
sitions relative to the low-spatial-frequency compo- 
nents. Our results show that this enhancement depends 
critically on the relationship between the SFR and sam- 
pling passband and that the preferred relationship is 
the one that maximizes the information capacity. 

4. The dynamic-range compression in the retina occurs 
before the signal is converted to the pulses that are 
transmitted to the visual cortex. A similar implemen- 
tation in the charge domain of the photosensor array, 
before a preamplifier reads the signal out to the analog- 
to-digital converter, could increase the effective dy- 
namic range by an order of magnitude or more with 
little loss of information while decreasing the power 
required for digital processing by several orders of 
magnitude. 
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